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1.  Introduction 


As  operational  demands  on  Soldiers  increase  and  with  multiple  new  display  technologies  hitting 
the  battlefield,  advanced  display  technology  will  be  needed.  Advances  include  multi-modal 
displays,  intelligent  automation,  and  (of  interest  here)  scalable  technology.  In  the  fullest  sense, 
scalability  is  the  tailored  reception  and  transmission  of  mission-essential  information  at  the 
appropriate  level  for  the  Soldier  and  in  the  appropriate  form.  A  key  design  goal  of  any  military 
system  is  to  maximize  the  survivability  and  lethality  of  the  Soldier-system  in  order  to  ensure 
mission  success.  Scalable  design  goes  a  step  further  and  can  be  described  as  the  synergistic 
interaction  of  the  hardware  equipment  requirements,  the  seamless  integration  of  distributed 
software  agents,  and  for  the  Soldier,  standardized  system  interactions  and  appropriate  cognitive 
workload,  which  in  turn  support  lowered  learning  requirements  and  increased  situational  aware¬ 
ness.  This  notion  of  scalability  is  especially  relevant  for  the  dismounted  Soldier  who  will  be 
controlling  unmanned  vehicles  (UVs),  will  need  to  maintain  his/her  local  security,  and  will 
complete  routine  tasks  as  the  battlefield  changes.  The  type  of  UV  controlled  from  that  display 
may  change,  depending  on  the  mission  or  environment.  The  dynamic  nature  of  the  Soldier’s 
environment  suggests  that  the  display  should  be  scalable,  which  would  minimize  the  Soldier’s 
need  to  be  familiar  with  multiple  system  hardware/software  configurations.  As  displays  are 
scaled,  they  should  have  minimal  learning  time,  acceptable  performance  metrics  for  completing 
required  tasks,  low  error  rates,  and  high  user  satisfaction  (Schneiderman,  1987). 

The  Robotics  Collaboration  Army  Technology  Objective  (RC  ATO)  is  addressing  the  scalability 
issue.  The  RC  ATO  is  a  joint  effort  by  the  Tank  Automotive  Research,  Development  and  Engi¬ 
neering  Center  (TARDEC)  and  the  U.S.  Army  Research  Laboratory  (ARL).  Under  this  program, 
TARDEC  and  ARL  are  working  to  develop  a  common  user  interface  for  robotic  control  that 
maximizes  multi-functional  Soldier  performance  of  primary  mission  tasks,  minimizes  unique 
training  requirements  by  minimizing  and  standardizing  required  interactions  and  managing 
workload  in  the  control  of  ground  and  air  unmanned  systems.  We  are  developing  and  evaluating 
scalable  interface  concepts  that  facilitate  the  Soldiers’  use  of  their  robotic  (i.e.,  unmanned) 
assets.  TARDEC  designed  displays  for  dismounted  applications  with  UVs.  The  dismounted 
displays  provided  basic  tele-operation  and  mission  planning  capability.  The  capabilities  of  the 
displays  were  modeled  on  the  existing  Vetronics  Technology  Integration  (VTI)  crew  station  with 
modifications  to  support  scalability.  If  the  new  versions  of  the  displays  meet  scalability  criteria, 
there  should  be  minimal  training  requirements  on  the  new  system  for  an  experienced  operator. 
Further,  performance  should  be  at  acceptable  limits,  and  user  satisfaction  should  be  high. 

Our  goal  was  to  provide  an  assessment  of  the  scaled  displays.  We  focused  our  assessment  on 
three  components  of  the  display:  display  size,  button  features  (e.g.,  size,  location),  and  the 
controllers  for  tele-operation.  In  addition,  we  evaluated  whether  the  displays  supported  the 


1 


requirements  of  scalability,  minimal  training,  and  user  satisfaction.  To  this  end,  trained  VTI 
operators  participated  in  this  assessment,  and  objective  performance  and  subjective  feedback 
were  obtained.  The  procedures  used  in  this  assessment  provided  a  means  for  capturing 
information  that  TARDEC  could  use  to  build  their  displays  and  provide  a  foundation  for 
our  scientific  research.  This  report  presents  the  findings  and  suggestions  from  the  usability 
assessment  of  these  prototype  dismounted  displays  designed  by  TARDEC  at  Fort  Knox, 
Kentucky,  from  10  July  to  14  July  2006. 


2.  Display  Design 


The  dismounted  displays  were  versions  of  the  mounted  crew  station,  the  VTI  display,  modified 
to  support  scalability.  The  modifications  implemented  to  support  scalability  were  the  ability  to 

•  Run  on  multiple  operating  systems,  graphics  systems,  and  hardware  platforms; 

•  Not  only  stretch/shrink  user  interface  control  features  but  to  also  change  the  features  into  a 
form  more  appropriate  for  the  intended  display  size; 

•  Configure  a  user  interface  to  take  advantage  of  target-specific  input/output  devices  (e.g., 
audio  output,  speech  input,  touch  screens,  global  positioning  system  [GPS]  units);  and 

•  Adapt  an  interface  based  on  mission-specific  needs  (e.g.,  the  tele-operation  screen  may 
increase  in  complexity  and  have  more  features  for  an  off-road  mission). 

TARDEC  created  the  displays  using  the  following  approach: 

•  Identified  the  tasks  that  need  to  be  accomplished  when  the  display  is  used; 

•  Defined  a  software  language  in  which  a  user  interface  can  be  described  while  being 
completely  device  independent;  this  means 

-  No  use  of  pixel  positions  or  sizes, 

-  Using  relative  positions  to  group  display  features  (e.g.,  buttons), 

-  Associating  function  with  display  features  so  that  informed  decisions  can  be  made  in 
regard  to  adapting  to  missions, 

-  Prioritizing  widgets  so  that  low  importance  features  can  be  dropped  or  placed  on  sub¬ 
screens,  and 

-  Using  a  human-readable  language  such  as  extensible  markup  language. 

•  Created  a  toolkit  that  encapsulates  the  concepts  described  in  the  interface  language; 

•  Created  utilities  that  can 
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-  Process  the  interface  description  language  into  a  database, 

-  Process  style  configuration  into  a  database,  and 

-  Process  configuration  into  a  database  with  rules. 

•  Created  a  software  scalability  component  to  determine  target  hardware  capability  and  adapt 
or  scale  the  interface  description  to  the  hardware,  taking  advantage  of  hardware-specific 
input  and  output  devices; 

•  Designed  and  implemented  a  new  user  interface  builder  tool  that  can  be  used  to 

-  Generate  the  interface  description, 

-  Display  the  interface  as  it  would  appear  in  various  targets, 

-  Define  widget  attributes,  and 

-  Generate  user  interface  style  configuration  file;  this  allows  the  interface  to  be  displayed 
in  various  differing  styles. 

For  this  assessment,  the  software  was  designed  so  that  it  could  adapt  to  certain  constraints,  the 
size  of  the  hardware  (i.e.,  display  size)  and  the  hardware  input  (i.e.,  control  modality  for  tele¬ 
operation).  The  scalable  software  was  programmed  to  run  on  two  displays: 

1.  Large  Tablet  Display  (Display  1) 

Model:  Xplore  Technologies:  iX104C2  display  (see  figure  1) 

Intel  Pentium  M  733  -  1.1  GHz 

10.4-inch  extended  graphic  array  transmissive  liquid  crystal  display 

Display  Size:  8.25  by  6.25  inches 

Button  Size:  2.5  cm  (wide)  by  2.0  cm  (high) 


Figure  1.  iX104C2  tablet  PC. 
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2.  Small  Personal  Digital  Assistant  (PDA)  Display  (Display  2) 

Model:  Transmeta  Cursoe  Model  TM5800  (see  figure  2) 

1-GHz  mobile  processor 
Display  Size:  5  by  3.75  inches 
Button  Size:  Two  sizes 

Small  Buttons  -  1 .5  cm  (wide)  by  1 .0  cm  (high)  (0  cm  spacing) 

Scaled  down  large  buttons  -  1.5  cm  (wide)  by  1.3  cm  (high)  (0.2  cm  spacing) 


Figure  2.  Transmeta  Crusoe  model  TM5800. 

In  addition  to  designing  the  software  so  that  it  ran  on  various  display  sizes,  TARDEC  implemented 
a  discovery  service  feature.  The  display  software  searches  the  network  for  available  features  that 
can  be  used  to  control  unmanned  assets.  The  discovery  service  feature  is  a  way  the  interface  can 
adapt  to  the  current  mission  or  robotic  asset.  The  discovery  service  feature  that  was  assessed  at 
Fort  Knox  was  a  tele-operation  feature.  UVs  can  be  tele-operated  with  a  traditional  joystick  (see 
figure  3).  TARDEC  designed  an  alternative  to  the  traditional  physical  joystick,  a  software-based 
joystick  (see  figure  4).  The  software-based  joystick  was  presented  on  the  display  and  the  operators 
used  a  stylus  or  finger  to  tele-operate  the  UV  (i.e.,  TALON1).  The  discovery  service  feature  for 
tele-operation  worked  in  the  following  manner.  If  the  physical  joystick  was  plugged  into  the 
display,  then  the  software-based  joystick  was  longer  functional  and  was  removed  from  the  display. 
If  the  physical  joystick  was  removed,  the  software-based  joystick  became  an  active  component  on 
the  display  and  the  Soldier  used  it  for  tele-operation. 


1  TALON,  which  is  not  an  acronym,  is  a  registered  trademark  of  Foster-Miller,  Inc. 
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Figure  3.  Physical  joystick;  Logitech2  Attack  3  joystick. 


Figure  4.  Software-based  joystick. 

It  is  worth  noting  that  the  two  dismounted  displays,  the  tablet  and  the  PDA,  were  designed  to  be 
as  similar  as  possible  to  the  mounted  VTI  crew  station  design;  however,  there  are  several  technical 
differences: 

1.  VTI  has  three  large  portrait  screens,  each  with  a  1200  x  1600  resolution;  dismount  has  two 
smaller  form  factors 


2Logitech  is  a  registered  trademark  of  Logitech,  Inc.  The  Logitech  logo  and  the  Logitech  products  referred  to 
herein  are  the  trademarks  or  the  registered  trademarks  of  Logitech. 
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a.  Tablet  1024x768 

b.  PDA  800x600 

2.  VTI  is  capable  of  displaying  two  interfaces  per  screen  with  the  ability  to  stretch  the  map 
interfaces  to  fill  a  full  screen;  dismount  can  only  display  a  single  interface  at  any  one  time. 

3.  There  is  only  a  single  interface  “look”  for  VTI;  dismount  has  multiple  “looks”  based  on 
target  and  configuration. 

a.  VTI  Look  where  the  buttons  and  gauges  are  the  same  as  VTI 

b.  VTI  Scaled  Look  where  buttons  and  gauges  are  the  same  as  VTI  but  are  scaled  down  to 
fit  a  smaller  display. 

c.  Small  Display  Look  where  the  buttons  and  gauges  are  reformatted  to  fit  a  smaller 
display  better. 

4.  There  is  only  a  single  menu  system  “feel”  for  VTI  which  uses  a  slide-out  system  that 
covers  the  interface  to  display  submenus;  dismount  has  multiple  “feels”  based  on  the  target 
and  configuration. 

a.  For  the  large  display,  the  menu  is  displayed  on  the  sides  as  in  VTI,  but  submenus 
replace  the  main  menu  and  never  cover  the  main  interface. 

b.  For  the  small  display,  the  menu  is  displayed  as  a  single  top  bar  with  pull-down 
submenus  that  cover  the  main  interface. 

5.  VTI  has  a  physical  yoke  for  tele-operation  of  assets;  dismount  can  use  either  a  software- 
based  joystick  (unique  to  the  dismount  display)  or  it  can  use  a  physical  joystick  if 
available;  the  software-based  joystick  is  disabled  and  the  icon  disappears  from  the 
interface. 

6.  VTI  has  “high  end”  hardware 

a.  More  powerful  processor, 

b.  Lots  of  memory  >1  GB, 

c.  More  powerful  graphics  card  and  texture  memory. 

Dismount  has  tablet-based  hardware  of  lesser  capability  than  the  VTI 

a.  Celeron  processor  (tablet)  and  Caruso  processor  (personal  digital  assistant  [PDA]) 

b.  Limited  memory  512  MB 

c.  Basic  graphics  card  capability,  minimal  texture  memory. 
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3.  Assessment  of  the  Scalable  Displays 


3.1  Participants  and  Procedures 

Twelve  male  Soldiers  at  Fort  Knox  participated  in  an  assessment  of  the  scaled  TARDEC  display 
designs  and  software  features.  The  focus  of  the  assessment  was  not  simply  the  question  of  if  the 
Soldier  completed  a  particular  task  using  the  particular  display  but  how  the  Soldier  was  ultimately 
able  to  accomplish  the  task.  Each  Soldier  completed  two  tasks,  a  route-planning  task  and  a  tele¬ 
operation  task. 

The  route -planning  task  was  completed  three  times,  once  for  each  display  configuration.  Figures  5 
through  7  show  the  displays  and  screen  layouts.  Displyay  configurations  for  route  planning  were 

1.  Display  1  (tablet)  with  the  large  buttons  -  2.5  cm  (wide)  by  2.0  cm  (high); 

2.  Display  2  (PDA)  with  small  buttons  -  1.5  cm  (wide)  by  1.0  cm  (high)  (0  cm  spacing); 

3.  Display  2  (PDA)  with  scaled  down  large  buttons  -  1.5  cm  (wide)  by  1.3  cm  (high)  (0.2  cm 
spacing); 

For  the  route-planning  task,  Soldiers  were  asked  to 

1 .  Plan  a  cross-country  route  between  four  points  on  a  map; 

2.  Locate  an  asset  and  assign  the  plan  to  the  asset; 

3.  Change  the  map  view  and  zoom  in  and  out  on  various  map  locations. 


Figure  5.  Large  display  (display  1)  with  map  view  for 
route  planning. 
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Figure  6.  Small  display  (display  2  with  small  buttons) 
with  map  view  for  route  planning. 


Figure  7.  Large  display  (display  1)  with  tele-operation  view. 

The  tele-operation  task  was  only  used  in  conjunction  with  Display  1  (see  figure  7).  The 
hardware/software  formats  for  tele-operation  were  Display  1  with  the  physical  joystick  and 
Display  1  with  software -based  joystick.  The  Soldiers  tele-operated  a  small  unmanned  ground 
vehicle  (UGV),  TALON  (see  figure  8).  The  TALON  robot  is  a  small  lightweight  tracked  UGV 
developed  by  Foster-Miller.  The  Soldiers  drove  the  TALON  in  a  “figure  eight”  pattern  around  a 
set  of  orange  cones  with  the  physical  joystick.  After  one  figure  eight  was  completed,  the 
physical  joystick  was  removed  and  the  Soldier  tele-operated  the  TALON  in  the  same  pattern 
using  the  software-based  joystick.  The  tele-operation  task  was  completed  with  the  TALON  in 
the  Soldier’s  line  of  sight  and  in  non- line  of  sight. 
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Figure  8.  TALON  small  UGV. 

3.2  Data  Collection  and  Reduction 

The  data  collected  were  self-reported.  The  Soldiers  were  queried  as  they  were  using  the  different 
displays;  their  opinions  about  the  displays  and  their  thought  processes  about  how  to  complete  the 
task  were  recorded.  Experimenters  used  knowledge  elicitation  techniques  focusing  on  usability 
heuristics  to  understand  how  the  Soldier  navigated  through  each  display  to  accomplish  the  tasks. 
Refer  to  appendix  A  for  examples  of  the  specific  questions.  Video  was  recorded  throughout  the 
assessment. 

The  data  were  coded  with  the  use  of  the  Skills,  Rules,  and  Knowledge  Framework  (Rasmussen, 
1986;  Reason,  1990)  as  commonly  applied  to  Ecological  Interface  Design  (Bums  &  Hajdukiewicz, 
2004;  Christoffersen,  1996).  Skills  are  defined  as  efforts  that  use  little  or  no  conscious  control 
after  intention  is  established  (Rasmussen,  1990).  In  the  context  of  the  operation  of  a  UGV 
interface,  a  procedure  that  was  conducted  at  the  skill  level  of  abstraction  might  take  the  form  of  an 
experienced  Soldier-operator  rapidly  selecting  icons  and  selecting  the  toggle  button  to  activate 
them,  all  without  diverting  his/her  attention  from  the  primary  task  of  maintaining  situational 
awareness  with  regard  to  the  UV  and  its  environment.  The  defining  characteristic  of  this  level 
of  operation  is  that  the  actual  manual  skill  of  selecting  and  activating  the  icons  is  performed 
without  cognitive  effort.  The  Soldier  makes  a  decision  to  perform  an  action  and  then  executes  the 
action  without  further  conscious  thought.  Performance  of  procedural  tasks  usually  reaches  the 
skill  level  when  operators  repeatedly  peform  the  task  until  it  becomes  automatic  and  no  longer 
requires  cognitive  effort  to  execute  after  the  decision  to  enact  the  task  is  made. 

Rule  level  processing  consists  of  the  use  of  relatively  simple  sets  of  rales  and  procedures  to 
select  a  course  of  action  in  a  familiar  setting  (Rasmussen,  1990).  The  use  of  a  rule  does  not 
require  underlying  knowledge  of  the  system,  only  knowledge  of  the  procedure.  Rule  use 
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generally  requires  a  moderate  and  intermittent  level  of  cognitive  resource  use  since  some 
recollection  and  tactical  control  over  the  various  elements  of  the  procedure  are  maintained. 

For  the  UGV  operator,  rules  are  likely  to  represent  regularly  used  procedures.  For  instance, 
experienced  Soldier-operators  might  know  that  when  they  experience  lag  between  the  control 
system  and  robot  operations,  one  easy  solution  is  to  turn  off  the  control  station  and  then  turn  it 
back  on.  This  represents  a  condition-action  pair  but  not  one  that  is  likely  to  be  used  commonly 
enough  to  reach  the  skill  level  of  abstraction.  Accordingly,  when  an  operator  has  used  a 
procedure  often  enough  to  use  it  as  a  standard  strategy  for  addressing  a  problem  or  system  state 
but  not  often  enough  for  its  use  to  have  become  automatic,  it  is  likely  to  be  represented  as  a  rule. 

Knowledge-based  processing  is  a  more  advanced  level  of  reasoning  used  when  a  novel  or 
unexpected  situation  is  encountered  (Wirstad,  1988).  Effective  use  of  information  at  this  level 
requires  a  fundamental  understanding  of  the  underlying  principles  or  laws  by  which  a  system  is 
operated  (Vicente,  1999).  If  perceived  understanding  of  the  system  is  incongruent  with  true 
system  behaviors,  then  a  series  of  troubleshooting  and  problem-solving  skills  will  be  applied. 
Thus,  an  experienced  Soldier-operator  encountering  a  new  problem  or  executing  an  unfamiliar 
function  for  the  first  time  will  probably  try  to  reason  his  or  her  way  through  the  process,  based 
on  familiarity  with  other  functions  of  the  system  or  using  other  known  procedural  functions  as  a 
model.  In  the  absence  of  the  kind  of  deep  knowledge  necessary  to  drive  this  knowledge-based 
strategy,  a  serial  process  of  finding  the  best  applicable  rule  through  trial  and  error  might  also  be 
applied.  Knowledge-based  processing  requires  a  very  high  levels  of  cognitive  resource  use, 
regardless  of  the  strategy  chosen,  because  each  step  in  the  procedure  needs  to  be  effortfully 
planned,  executed,  and  monitored. 

The  Skills,  Rules  and  Knowledge  framework  (Rasmussen,  1986;  Reason,  1990)  was  applied  to 
the  data  set  as  described  in  the  following  categories: 

1 .  Did  the  Soldier  complete  the  task? 

2.  Did  the  Soldier  ask  clarifying  questions  about  the  task? 

The  Soldiers  were  able  to  ask  the  experimenters  questions  throughout  the  experiment.  The 
questions  were  asked  because  the  Soldier  did  not  know  how  to  complete  a  single  step  or  series  of 
steps  needed  to  accomplish  the  task.  If  questions  were  asked,  it  was  inferred  that  the  Soldiers  did 
not  reach  a  skill  level  on  that  portion  of  the  task. 

3.  Did  the  Soldiers  use  their  primary  training  on  the  VTI  system  to  complete  this  task? 

We  were  interested  in  whether  Soldiers  were  able  to  use  knowledge  transfer  from  the  original 
VTI  design  in  order  to  complete  the  entire  task  with  the  dismounted  displays.  If  the  Soldiers 
could  use  the  displays  easily  and  did  not  require  help  from  the  experimenters,  it  was  assumed 
that  the  VTI-learned  skills  or  rules  transferred  between  displays. 

4.  Did  the  Soldier  use  a  knowledge-based  (i.e.,  prior  experience)  solution  to  complete  the 
task? 
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If  perceived  understanding  of  the  system  is  incongruent  with  true  system  behaviors  (i.e.,  VTI 
display  had  a  different  layout  than  the  dismounted  displays),  then  a  series  of  troubleshooting  and 
problem-solving  skills  would  be  applied,  such  as  trial  and  error  or  serial  search. 

5.  Was  the  Soldier  able  to  confirm  that  an  action  was  taken?  Was  feedback  provided  to  the 
Soldier  which  could  confirm  that  s/he  had  made  a  change  in  the  robot? 

This  represents  the  number  of  Soldiers  who  stated  that  they  had  a  high  level  of  confidence  they 
had  made  changes  in  the  asset. 


4.  Results 


The  results  are  presented  as  the  mean  number  of  individuals  who  responded  accordingly  in  the 
categories  described  in  the  previous  section. 

4.1  Performance  Route  Planning 

Task  1 :  Soldiers  were  asked  to  draw  a  cross-country  route  between  four  Xs  on  the  map  and  to 
use  a  starting  point,  two-way  points,  and  an  end  point. 

Results:  Number  of  Soldiers  who 

•  Completed  this  task:  12  (100%) 

•  Asked  clarifying  questions  about  the  task:  12  (100%) 

•  Used  primary  skills/rules  training  to  complete  this  task:  0  (0%) 

•  Used  a  knowledge-based  solution  to  this  task:  12  (100%) 

•  Could  confirm  assignment  of  plan  asset  (Feedback):  NOT  APPLICABLE 
Task  2:  Soldiers  were  asked  to  locate  an  asset  and  assign  their  plan  to  the  asset: 

Results:  Number  of  Soldiers  who 

•  Completed  this  task:  12  (100%) 

•  Asked  clarifying  questions  about  the  task:  7  (58%) 

•  Used  primary  skills/rules  training  to  complete  this  task:  0  (0%) 

•  Used  a  knowledge-based  solution  to  this  task:  12  (100%) 

•  Could  confirm  assignment  of  plan  asset  (Feedback):  0  (0%) 

Task  3:  Soldiers  were  asked  to  change  the  map  types  and  zoom  in  and  out  of  the  map. 

Results:  Number  of  Soldiers  who 
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•  Completed  this  task:  12  (100%) 

•  Asked  clarifying  questions  about  the  task:  12  (100%) 

•  Used  primary  skills/rules  training  to  complete  this  task:  0  (0%) 

•  Used  a  knowledge-based  solution  to  this  task:  12  (100%) 

•  Could  confirm  assignment  of  plan  asset  (Feedback):  NOT  APPLICABLE 

4.2  Performance  of  T ele-operation  Task 

Because  of  technical  difficulties  with  the  TALON  (battery  power  and  network  connectivity  to 
display),  several  Soldiers  were  not  able  to  complete  the  tele-operation  task  with  both  joysticks. 
General  observations  made  by  the  experimenters  and  Soldier  comments  are  presented. 

4.2.1  Observations  Made 

4.2. 1 . 1  Physical  Joystick 

•  Soldiers  reported  that  traversing  the  obstacle  course  (particular  to  turning)  required  fine 
control,  which  was  not  available  with  the  physical  joystick. 

•  Soldiers  reported  physical  strain  from  holding  the  joystick.  They  preferred  a  built-in 
joystick. 

4.2. 1.2  Software-Based  Joystick 

•  The  software  joystick  afforded  the  use  of  a  precision  grip. 

•  The  software  joystick  was  useful  for  fine  movements. 

•  The  software  joystick  allowed  the  Soldiers  to  set  a  constant  speed. 

•  Soldiers  reported  the  software-based  joystick  was  useful  for  navigation  through  turns. 

•  Soldiers  reported  that  the  visual  representation  of  speed  as  concentric  circles  was  useful  for 
proper  controlling  movement  of  the  robot.  This  feature  was  not  available  on  the  physical 
joystick. 

•  With  the  software  joystick,  it  took  time  for  the  Soldier  to  learn  how  to  adjust  the  UV  speed, 
especially  during  the  initial  learning  that  led  to  variable  UV  control. 

4.3  General  Ergonomic  Findings 

The  experimenters  observed  the  Soldiers  using  the  displays  and  asked  ergonomics-related 
questions  and  obtained  subjective  feedback  about  how  difficult  a  location  was  to  press  on  each 
display,  the  usability  of  buttons  and  menus,  and  preference  for  the  different  hand  grips. 
Experimenters  asked  the  Soldiers’  opinions  about  using  the  displays  in  various  operational 
settings  (i.e.,  mounted  or  dismounted  operations),  given  the  size  and  weights  of  the  displays. 
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4.3.1  General  Ergonomic  Findings  for  Display  Size 

4.3. 1 . 1  Large  Display  (tablet) 

Size  -  8.25  by  6.25  inches 
A.  Observations 

•  Could  not  be  used  in  a  dismounted  situation. 

•  Was  easiest  to  use  when  tilted  backward  at  least  15  to  20  degrees. 

•  Afforded  the  use  of  thumb  presses  for  button  activation. 

o  Thumbs  could  not  be  accurately  used  in  center  of  screen. 

o  The  use  of  thumbs  on  lower  parts  of  the  screen  caused  the  Soldier  to  take  his 
grip  off  the  display. 

•  Stylus  could  not  be  used  if  screen  was  not  tilted  (i.e.,  unnatural  writing  position). 

4. 3. 1.2  Small  Display  (PDA) 

Size  -  5  x  3.75  inches 

A.  Observations 

•  Afforded  the  use  of  a  chuck  grasp  with  the  left  hand  because  of  the  grips  on  the 

display. 

•  Neither  thumb  could  be  used  for  small  buttons  if  Soldier  had  average  to  large 

fingers. 

•  Without  researcher  instruction,  the  Soldiers  always  used  the  right  index  finger. 

•  Left  thumb  could  not  be  used  to  hit  lower  left  buttons  because  of  reach  and 
contortion  required. 

B.  Overall  Observations  (regardless  of  display  size) 

•  Right  hand  was  used  more  often  than  left  hand. 

•  Right  hand  often  obscured  the  display  when  crossing  over  to  use  left  side  buttons. 

4.3.2  General  Ergonomic  Findings  for  Button  Size 

4.3.2. 1  Large  Display  (tablet) 

With  large  buttons  -  2.5  cm  by  2.0  cm;  buttons  on  left  and  right  sides  of  the  display 
A.  Observations 

•  As  the  Soldier  moved  farther  away  from  the  sides  of  the  display,  the  thumbs  were 
more  difficult  to  use. 
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•  Soldiers  always  switched  from  thumb  to  index  finger  to  place  route  points  on  the 
inside  of  the  screen. 

•  Thumbs  could  not  be  used  to  touch  buttons  on  the  bottom  of  the  display  when  the 
display  had  to  be  held.  This  is  because  of  an  unnatural  movement  of  the  thumb. 

•  Right-handed  people  were  even  less  accurate  with  the  left  thumb  than  those  who 
were  left-handed. 

•  Soldiers  had  a  difficult  time  hitting  the  smaller  buttons  on  the  small  display.  This 
is  because  it  required  precision  to  hit  buttons  surrounded  by  other  buttons. 

4. 3. 2.2  Small  Display  (PDA) 

A.  With  small  buttons  (1.5  cm  by  1.0  cm);  buttons  on  top  of  the  display 

•  100%  of  Soldiers  leaned  into  the  display  to  read  the  buttons. 

•  Soldiers  used  the  stylus  with  the  smaller  buttons. 

•  Smaller  buttons  were  easier  to  press  at  the  comers  of  the  screen  but  were  harder  to 
press  as  the  buttons  cascaded. 

B.  With  scaled  down  large  buttons  (1.5  cm  by  1.3  cm);  buttons  on  left  and  right  sides  of 
the  display 

•  Soldiers  preferred  the  size  of  these  buttons  to  the  larger  buttons  used  on  the  large 
display  and  the  small  buttons  used  on  the  small  display. 

•  Soldiers  had  a  very  difficult  time  using  thumbs  over  left  side  buttons  when  the  left 
hand  was  in  the  hand  grip  strap  on  the  display. 

C.  Overall  Observations 

•  The  large  buttons  (2.5  cm  x  2.0  cm)  afforded  the  use  of  fingers. 

•  When  the  display  had  a  hand  grip,  the  buttons  afford  the  use  of  thumbs. 

•  Soldiers  preferred  the  smaller  form  factor  because  of  the  improved  visual  contrast 
and  minimized  amount  of  information  on  the  display. 


5.  Discussion  and  Recommendations 


The  purpose  of  this  usability  assessment  was  to  evaluate,  through  Soldier  feedback  and  experi¬ 
menter  observation,  the  functionality  of  specific  display  features  for  control  of  UVs.  The  displays 
designs  were  modified  from  the  VTI  crew  station.  The  intent  was  to  design  the  smaller  displays  so 
that  a  Soldier  could  easily  transition  from  using  a  larger  version  of  a  display  to  a  smaller  version 
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and  vice  versa.  In  our  assessment,  we  investigated  whether  Soldiers  could  make  this  transition  as 
well  as  assess  features  of  the  displays.  In  this  section,  we  discuss  the  findings  of  this  assessment 
and  the  implications  for  future  designs. 

5.1  Summary  of  Task  Performance 

Overall,  the  Soldiers  were  successful  in  using  the  various  display  configurations  to  plan  a  route  for 
a  UV.  This  suggests  that  the  displays  preserved  the  basic  functionality  originally  designed  in  the 
VTI  crew  station  for  completing  route-planning  tasks.  The  implementation  of  the  functionality, 
however,  was  not  sufficient  for  the  Soldiers  to  be  able  to  complete  the  tasks  without  assistance 
from  the  experimenters.  All  the  Soldiers  had  to  ask  clarifying  questions  about  Tasks  1  (plan  a  path 
between  four  points)  and  3  (change  the  map  view).  Further,  more  than  half  of  the  Soldiers  asked 
for  clarification  of  Task  2  (assign  the  plan  to  an  asset).  These  results  suggest  that  the  original 
display  design  lacked  design  principles  that  were  transferable  to  smaller  display.  For  example, 
Task  1  required  users  to  navigate  through  the  display  and  create  a  new  route.  Even  though  the 
labels  and  sequence  of  events  remained  the  same,  the  locations  of  the  buttons  for  this  task  were 
different  than  the  original  VTI  design.  The  Soldiers  indicated  that  they  were  also  unsure  where  to 
go  to  assign  a  plan  to  an  asset.  It  was  not  easily  distinguishable  as  to  why  some  accomplished  this 
task  with  less  difficulty  than  others.  When  we  questioned  the  Soldiers  who  completed  this  task 
without  asking  a  question,  the  primary  answer  was  “/ just  read  all  of  the  buttons  and  this  one  made 
the  most  sense .”  Uncertainty  about  the  functionality  of  buttons  and  the  associated  violation  of  user 
expectancies  for  the  button  locations  may  have  caused  the  error  rate  on  the  various  components  of 
the  route-planning  task. 

Through  observation  and  questioning,  we  gained  some  insight  into  how  the  Soldiers  completed 
the  tasks.  When  a  Soldier  seemed  to  be  stuck  in  the  process  of  completing  a  task,  we  asked, 

“what  are  you  looking  for?  “are  you  looking  for  any  key  words?  “is  this  similar  to  how  you 
completed  this  task  on  the  original  display?  "Please  explain.  “what  are  you  doing  right  now?  ” 
Through  the  use  of  this  type  of  questioning,  the  following  conclusions  were  drawn.  Most  of  the 
Soldiers  used  a  serial  search  tactic  to  accomplish  tasks.  The  Soldiers  read  every  button  until 
they  reached  the  button  that  matched  their  target.  Further,  most  Soldiers  started  in  the  upper  left- 
hand  comer  of  the  display.  Second  to  serial  search.  Soldiers  used  the  process  of  elimination. 
When  Soldiers  were  still  unsure  how  to  proceed,  they  read  every  button  on  the  screen  in  a  serial 
fashion  (left  to  right  and  top  to  bottom)  and  chose  the  button  that  made  the  most  sense.  The  final 
tactic  used  was  to  ask  the  experimenter  how  to  proceed.  In  this  case,  the  researcher  directed  the 
Soldiers’  attention  to  the  appropriate  button.  Typically,  after  the  redirection,  Soldiers  were  then 
asked,  “Would  you  expect  to  find  this  functionality  under  this  button?  “What  do  you  think 
about  that  button  functionality?  “How  did  you  do  this  in  your  original  training?  ” 

Our  usability  assessment  showed  that  some  knowledge  obtained  from  training  on  the  original 
VTI  crew  station  did  positively  affect  the  Soldiers’  ability  to  use  the  scalable  displays.  Generally, 
Soldiers  knew  what  the  displays  were  used  for  (controlling  a  robot  and  planning)  and  how  to  hold 
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and  interact  with  the  buttons  on  the  display.  The  buttons  seemed  familiar  to  them  but  out  of  place. 
What  did  not  transfer  were  the  locations  and  groupings  of  the  buttons  and  how  to  navigate  through 
menus. 

In  general,  Soldiers  had  been  trained  on  a  system  that  did  not  fully  afford  transfer  to  a  smaller 
display.  The  original  VTI  design  was  a  multi-display  system  consisting  of  three  screens.  Buttons 
on  the  screens  are  the  roughly  the  same  size,  shape,  and  color.  Soldiers  were  trained  how  to  use 
the  VTI  crew  station,  that  is,  the  location  and  function  of  each  button,  mission  planning,  and  asset 
management.  Soldiers  continued  to  train  on  the  system  until  they  were  confident  that  they  could 
perform  the  necessary  training  objectives.  Through  Soldier  observation  and  discussion,  we  made 
the  determination  that  what  made  the  VTI  crew  station  easy  to  use  was  time  and  effort  spent 
learning  the  new  system.  Time  and  effort  are  the  two  essential  components  for  effective  training 
(Schneider,  1985).  However,  the  goal  of  scalability  is  not  to  develop  training  programs  for  each 
display  but  to  develop  principles  and  guidelines  that  can  be  used  to  create  scalable  software  that 
requires  little  to  no  re-training.  We  want  learning  on  one  display  to  transfer  to  another  with  little 
cognitive  effort  by  the  Soldier. 

5.2  Design  Recommendations 

The  question  then  remains,  how  can  Soldiers  learn  to  use  the  new  displays?  The  research  of 
Rasmussen  (1993)  and  colleagues  into  skills,  rules,  and  knowledge  types  provides  a  good 
theoretical  framework  to  guide  scalable  display  design.  Experience  gained  through  training  is  a 
key  indicator  on  whether  a  skill,  rule,  or  knowledge -based  solution  should  be  used  (Hammond, 
1980;  Rasmussen,  1993).  A  skill  is  fast,  efficient,  and  well  practiced.  Skills  can  be  practiced 
with  the  observer  providing  minimal  cognitive  effort.  When  the  task  is  well  practiced  and  easy 
to  complete,  it  becomes  a  skill  that  does  not  require  the  user  to  engage  in  cognitive  effort  to 
complete  the  task. 

A  rule  requires  the  user  to  attend  to  the  stimuli  and  attach  meaning  to  those  stimuli.  It  can  provide 
guidance  toward  a  section  of  the  scalable  design  that  meets  the  user’s  needs  to  accomplish  the 
objectives.  One  way  to  establish  a  set  of  scalable  rules  is  through  the  use  of  features  such  as  color, 
shape,  and  location  of  buttons.  Color  can  be  used  as  a  redundant  cue  that  signifies  meaning  to  the 
user  (Wyszecki,  1986).  Shape  can  be  used  as  another  organizational  cue  that  signifies  meaning. 
The  user  could  perceive  different  shapes  as  having  different  functions  (e.g.,  “I  can  press  these 
buttons  but  not  buttons  of  that  shape”).  For  example,  a  rule  such  as  the  “color  blue  always  means 
menu”  could  be  attached  to  numerous  menu  buttons  that  have  different  text  signifying  different 
categories.  Although  skills  have  no  need  for  interpretation,  rules  can  be  interpreted  differently 
according  to  context.  Smaller  blue  buttons  that  signify  menus  should  be  selected  with  a  stylus 
while  larger  blue  buttons  can  be  used  with  a  finger. 

Button  locations  can  also  be  used  to  the  designer’s  advantage.  Buttons  that  are  going  to  be  used 
often  should  be  placed  at  the  beginning  of  menus  or  toward  the  top  of  a  menu.  A  primary  button 
that  is  used  often  should  have  space  surrounding  it  that  differentiates  it  from  other  buttons.  The 
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designer  can  determine  the  button  qualities  by  watching  users  interact  with  the  software 
(Lidwell,  Holden,  &  Butler,  2003,  Nielson,  1993;  Nielson  &  Mack,  1994). 

Where  the  user  expects  to  find  a  button  predicts  where  s/he  will  look  first  on  a  display.  Users 
expect  to  find  creation  buttons  such  as  “new”  at  the  beginning  of  menus  and  edit  buttons  farther 
along  in  the  menu  structure.  Buttons  at  the  top  of  menus  are  expected  to  be  used  more  often  than 
buttons  at  the  bottom  of  menus.  Users  will  often  scan  buttons  at  the  top  of  menus  before  looking 
at  the  bottom  of  a  menu. 

Objects  grouped  together  are  often  perceived  to  have  the  similar  or  related  functionality  (Wickens 
&  Carswell,  1995;  Tversky,  1977).  When  buttons  with  varying  functionality  are  grouped  together, 
the  group  as  a  whole  represents  an  organized  structure  to  the  human,  which  should  have  some 
common  meaning  though  even  though  it  does  not.  Further,  a  group  with  dissimilar  buttons  and  a 
main  button  that  does  not  describe  the  group  as  a  whole  will  cause  the  user  to  question  the  logic  of 
the  system  (figure  9).  A  user  often  gives  feedback  such  as  “I  really  don’t  think  these  buttons 
belong  here.”  For  example,  in  the  display  used  in  this  assessment,  the  main  menu  button  of  edit 
did  not  match  the  sequence  of  buttons  (new  plan,  add  points,  zoom,  and  select  plan)  that  followed. 
The  edit  button  lost  its  meaning  and  required  the  Soldier  to  memorize  the  lower  menu  buttons 
rather  than  group  them  together  by  functionality. 


Figure  9.  Inefficient  menu  hierarchy  system. 


Ineffective  button  grouping  increases  demands  on  working  memory.  Soldiers  have  to  remember 
the  specific  locations  of  single  buttons  and  not  the  location  of  a  group  of  buttons  that  have 
similar  functionality.  Memory  traces  are  more  likely  to  be  made  through  a  main  button  when  the 
functions  of  those  buttons  have  similar  meaning  captured  by  the  main  menu  button  (figure  10). 

It  is  more  likely  that  select  points,  add  points,  copy,  and  paste  will  be  associated  with  the  main 
button  of  edit  than  the  button  of  new  plan. 

The  next  level  of  information  processing  important  for  display  design  is  knowledge.  At  the 
knowledge-based  level,  information  reaches  the  Soldier’s  mind  and  does  not  match  an  existing 
skill-  or  rule-based  behavior;  a  Soldier  will  not  push  a  button  with  the  purposeful  intent  of 
accomplishing  the  task  (skill)  or  apply  a  code-based  behavior  response  and  look  for  a  sign  that 
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makes  sense  (rule).  In  this  case,  the  Soldier  does  not  have  any  rules  that  apply  or  any  skills  that 
are  relevant  to  the  new  task,  but  the  Soldiers  look  for  cues  and  integrate  them  into  their  mental 
model  of  how  things  should  or  likely  will  work.  In  order  to  meet  the  goals  of  improving  transfer 
of  training  and  scalable  design,  it  is  important  to  elicit  existing  mental  models  and  focus  on  design 
aspects  that  employ  rather  than  contradict  it.  For  example,  a  Soldier  in  the  assessment  stated  that 
groups  of  buttons  did  not  make  sense:  “We  either  plan  or  we  execute  in  the  Army;  that’s  how  we 
are  trained.  ”  The  reference  was  to  the  route-planning  task.  The  experimenter  wanted  the  Soldier 
to  assigned  a  plan  to  an  asset  and  have  the  asset  execute  that  plan.  The  Soldier  could  not  find  the 
assigned  button  and  stated  that  this  order  of  execution  should  be  clearly  marked.  In  the  Soldier’s 
mental  model,  planning  and  executing  are  tasks  that  should  be  clearly  marked  and  separated  in  the 
display.  Thus,  the  Soldier’s  interaction  with  the  system  was  hindered  because  the  display  did  not 
match  his  mental  model  about  how  action  and  planning  in  the  Army  are  presented. 


Figure  10.  Efficient  menu  hierarchy  system. 


In  designing  a  display  around  mental  models,  the  designers  need  a  common  reference  of  how  a 
Soldier  completes  a  task  in  the  component  sub-tasks.  The  best  way  to  understand  a  mental  model 
and  provide  a  functional  display  design  that  represents  this  model  is  through  user  feedback  and 
laboratory  testing.  Errors  are  likely  to  be  made  at  the  knowledge-based  level  because  the  Soldier  is 
problem  solving  where  to  go  (as  seen  during  the  assessment)  rather  than  simply  making  a  decision 
about  what  choice  will  definitely  be  made  next.  According  to  the  Skill,  Rule,  Knowledge  model,  a 
person  might  operate  at  the  knowledge-,  rule-,  or  skill-based  level  and  switch  between  them, 
depending  on  task  familiarity.  As  skills,  rules,  and  knowledge  relate  to  this  assessment,  we  should 
consider  the  following  for  future  design  efforts: 

1 .  During  this  assessment,  the  Soldiers  did  not  use  skill-based  behavior.  The  Soldiers  should 
show  highly  practiced  movements  in  a  nearly  effortless  behavior  in  navigating  through  the 
new  display. 

2.  The  Soldiers  did  not  use  a  system  of  rules  learned  in  the  original  display.  Design  principles 
such  as  color  of  buttons  or  menus,  or  grouping  by  space  or  location  of  buttons  and  menus 
that  have  similar  meaning  were  not  used  and  as  such,  could  not  meet  the  requirements  for 
rule-based  actions. 
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3.  Knowledge -based  problem  solving  occurs  when  a  human  encounters  a  new  situation, 
develops  a  strategy,  and  systematically  accomplishes  his  goal  through  problem  solving. 
When  asked,  almost  every  Soldier  stated  that  s/he  identified  a  key  word  s/he  was  looking 
for  and  then  read  everything  on  the  screen  in  order  to  accomplish  the  task  at  hand.  Even 
though  the  new  system  was  easy  to  read  and  had  symmetrical  buttons  that  were  similar  to 
the  original  system,  we  identified  certain  strategies  the  Soldiers  used.  These  strategies  are 
portrayed  in  the  Soldier  feedback. 

Soldier  comments  when  asked  to  perform  the  original  task: 

Experimenter  -  “What  are  you  thinking  right  now?  ” 

Soldier  -  “I  don ’t  know  where  any  buttons  are  ” 

Experimenter  -  “How  did  you  accomplish  that  task?  ” 

Soldier  -  “I’m  reading  all  of  the  buttons  ” 

Experimenter  -  “Where  do  you  start  reading?  ” 

Soldier  -  “I  start  at  the  top  left,  go  down,  go  up  the  top  right,  and  go  down  ” 

Soldier  -  “I  don ’t  know  what  buttons  to  hit  to  get  to  other  buttons  ” 

Soldier  -  “I  know  I’m  looking  for  a  plan  button,  if  it  wasn  ’t  in  the  first  I’d  go  to  the 
second,  then  the  third” 

It  is  likely  that  Soldiers  in  the  assessment  learned  the  original  display  system  to  a  skill-based 
level.  When  confronted  with  a  task  simply  go  to  “X”  position  on  the  three  display  system  and 
push  a  button  that  leads  to  “Y”  functions  or  capabilities,  they  were  successful.  However,  the 
skills  that  the  Soldiers  learned  through  training  are  not  likely  to  transfer  to  smaller  displays  used 
in  our  assessment  because  the  original  encoding  of  the  skills  relied  on  spatial  memory  of  where 
the  button  was  located,  not  the  button’s  color,  relationship  to  other  buttons,  categories  of  buttons, 
etc.  We  suggest  using  design  principles  described  as  rules  that  can  be  scaled  across  displays  of 
all  sizes  and  dimensions.  Rules  should  allow  the  Soldiers  to  use  decision-making  skills  and  not 
problem-solving  skills. 

5.3  Summary  of  Ergonomic  Findings 

Overall,  the  ergonomically  related  results  showed  that  the  small  display  with  the  smaller  buttons 
was  preferred  by  the  Soldiers  to  the  other  display  configurations  for  dismounted  operations.  For 
mounted  operations  or  during  conditions  when  the  Soldier  would  not  have  to  be  mobile,  the 
larger  display  was  preferred  by  the  Soldiers  we  interviewed.  Ergonomic  results  for  the  joysticks 
showed  that  the  chuck  grip,  such  as  that  on  the  physical  joystick,  is  not  sensitive  enough  for  a 
situation  when  precision  is  needed.  Precision  joysticks  such  as  the  software-based  joystick,  are 
better  in  these  conditions  and  those  when  the  Soldier  would  be  dismounted. 
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During  the  assessment,  the  researchers  observed  that  many  of  the  displays  afforded  the  use  of  the 
thumbs.  We  observed  that  Soldiers  took  more  time  to  touch  buttons  toward  the  center  of  the 
screen  and  were  less  accurate  in  doing  so.  Additionally,  several  Soldiers  were  asked  how  far 
they  could  comfortably  use  their  thumbs,  starting  from  the  resting  position  at  the  edges  of  the 
screen,  moving  toward  the  center.  After  approximately  three  touch  buttons,  or  approximately 
9  cm,  functionally  it  was  difficult  to  use  the  thumbs  on  the  display.  The  Soldiers  often  used  their 
right  index  fingers  to  complete  functions  outside  the  thumb’s  range,  such  as  completing  the 
sequence  of  button  presses  on  the  sliding  menu. 

Fitts’  law  can  provide  guidance  into  developing  system  specifications  of  width  and  depth  of  menus 
from  the  side/edges  of  the  display  where  the  Soldiers’  fingers  are  resting.  Fitts’  law  states  that  the 
smaller  and  more  distant  a  target,  the  longer  it  will  take  to  move  to  a  resting  position  over  the 
target  (Fitts,  1954;  Jagacinski,  1989).  Additionally,  the  faster  the  movement  has  to  be  and  the 
smaller  the  target,  the  more  likely  error  will  be  introduced.  Speed  and  target  size  dilemmas  in 
display  design  are  often  referred  to  as  the  “speed/accuracy  trade-off’.  On  the  small  displays,  the 
buttons  should  be  larger  and  have  more  space  between  them  if  they  are  to  be  used.  On  the  smaller 
displays,  designers  should  minimize  the  amount  of  text  information  presented  in  order  to  decrease 
the  reading  requirement  (a  difficult  task  in  dismounted  and  moving  conditions).  Further,  if  the 
display  has  a  left-handed  grip  and  affords  the  use  of  the  thumb,  then  buttons  should  not  be  placed 
on  the  bottom,  left-hand  side  of  the  display.  Future  research  should  investigate  hand  position, 
menu  width,  and  target  size  (button  size).  Based  on  the  current  data  and  observations,  the 
researchers  recommend  no  more  than  three  buttons  across  for  touch  screen  displays. 


6.  Future  Research 


For  future  research,  we  have  identified  four  tasks  to  guide  us  toward  achieving  our  goal  of 
designing  scalable  interface  concepts  that  facilitate  the  Soldiers’  use  of  their  robotic  assets  in 
dismounted  and  mounted  mission  environments: 

Task  1 :  Identify  critical  tasks  associated  with  scalable  displays.  Who  are  our  Soldiers  (users)  and 
what  are  they  trying  to  do?  Prioritize  these  tasks.  Examine  existing  display  designs  in  use  by 
military  organizations.  Specifically,  employ  display  designs  undergoing  development  within  the 
military  community.  Identify  chosen  technology  and  catalog  the  different  methods  of  user  input 
for  the  display.  Identify  the  environmental  conditions  in  which  the  display  can  be  operated  in 
order  to  help  determine  appropriate  functionality  (i.e.,  dismounted,  mounted). 

Task  2:  Determine  hierarchy  of  system  functionality  within  a  task,  based  on  mental  models  of 
Soldiers.  Take  a  task  that  will  be  included  in  the  scalable  display  and  research  any  available  task 
analyses,  looking  for  key  identifiers  of  function  which  should  be  included  in  the  menu  hierarchy. 
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Determine  the  between-task  prioritization  in  the  software  and  test  this  hierarchy  through  use  of 
design  principles  across  multiple  displays. 

Task  3:  Identify  what  makes  a  design  principle  fit  the  category  of  skill,  rule,  or  knowledge 
based.  Use  the  scientific  literature  to  gather  identifiers  of  what  is  a  skill,  rule,  and  knowledge 
trait  and  match  the  outcome  to  design  principles.  Determine  how  a  design  rule  can  become  skill 
or  knowledge  based.  The  goal  is  to  develop  design  rules  that  can  be  efficiently  scaled  across 
displays,  minimizing  the  need  for  additional  training. 

Task  4:  Create  a  “testable”  function  of  what  is  acceptable  as  a  scalable  design  principle.  Take  a 
design  principle  across  multiple  display  platforms  and  look  at  increases  or  decreases  in  Soldier 
performance.  Separate  principles  into  categories  of  minimally  acceptable  and  optimal  task 
performance. 
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Appendix  A.  Questionnaire  Template  for  Route  Planning  Task 


Route  Planning  Tasks 

1 .  Draw  a  cross  country  route  between  these  4  x’s  using  a  start  point,  two  way  points  and  an 
end  point. 

a.  Insert  a  point  into  the  plan 

Experimenter  Questions: 

What  are  you  thinking  about  right  now? 

What  comes  to  mind  when  I  asked  you  to  create  a  new  route? 

What  did  you  expect? 

What  would  you  want  to  be  there? 

(For  new  plan)-  Is  that  where  you  expected  it  to  be? 

(If  lost  completely)  -  What  were  you  thinking  about? 

What  should  it  be  like  or  similar  to  (i.e.,  other  programs  that  you  have  used)? 

How  did  you  feel  about  the  mission  button  layouts? 

2.  Locate  an  asset.  Assign  your  plan  to  your  asset. 

Experimenter  Questions: 

What  comes  to  mind  when  I  asked  you  to  assign  a  plan? 

What  should  you  be  looking  for  that  makes  sense  to  you? 

Did  you  know  where  to  go  to  find  an  asset? 

3.  Change  the  map  types. 

a.  Zoom  in  and  out  on  the  map  type  you  changed  to. 

Experimenter  Questions: 

Were  there  any  buttons  that  you  thought  were  critical? 

What  would  those  buttons  be? 

What  do  think  about  the  button  size? 

What  do  think  about  the  button  layouts? 

What  did  you  think  about  the  colors? 

What  do  you  think  about  the  fonts? 

Objective  Metrics: 

How  many  times  did  they  show  frustration? 

How  many  times  did  they  choose  the  wrong  button? 

How  many  times  did  they  ask  for  help? 
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Subjective  Feedback: 

Which  display  was  easiest  to  use  (Rank  order  them  please)? 

1. 

2. 

3. 

Why  1st? 

Why  2nd? 

Why  3rd? 

Rank  order  the  displays  for  mounted  missions.  Assume  you  have  to  carry  the  display. 

1. 

2. 

3. 

Rank  order  the  displays  for  dismounted  missions.  Assume  you  have  to  carry  the  display. 

1. 

2. 

3. 
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6501  E  1 1  MILE  RD  MAIL  STOP  284 
BLDG  200 A  2ND  FL  RM  2104 
WARREN  MI  48397-5000 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MF  C  HERNANDEZ 
2421  NW  AUSTIN  RD  STE  220 
FORT  SILL  OK  73503-9042 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MW  E  REDDEN 
BLDG  4  ROOM  332 
FT  BENNING  GA  31905-5400 

ARMY  RSCH  LABORATORY  -  HRED 
ATTN  AMSRD  ARL  HR  MN  R  SPENCER 
DCSFDI  HF 

HQ  USASOC  BLDG  E2929 
FORT  BRAGG  NC  28310-5000 

ARMY  G1 

ATTN  DAPE  MR  B  KNAPP 

300  ARMY  PENTAGON  ROOM  2C489 

WASHINGTON  DC  20310-0300 


ABERDEEN  PROVING  GROUND 
1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  Cl  OK  (TECH  LIB) 
BLDG  4600 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  Cl  OK  TP  S  FOPPIANO 

BLDG  459 

1  DIRECTOR 

US  ARMY  RSCH  LABORATORY 

ATTN  AMSRD  ARL  HR  MR  F  PARAGALLO 

BLDG  459 

5  DIRECTOR 

US  ARMY  RSCH  LABORATORY 
ATTN  AMSRD  ARL  HR  SE  K  COSENZO 
BLDG  459 


28 


